Mobile augmented reality system

ABSTRACT

Systems, apparatuses and methods to provide image data, augmented with related data, to be displayed on a mobile computing device are disclosed. An example mobile device includes a camera to provide images of a scene from different angles to a server, at least one sensor to sense a position and an orientation of the camera, and a screen to present augmented reality data over the scene based on the position and the orientation of the camera and on a three-dimensional representation of the scene based on the images.

PRIORITY

This patent is a continuation of and claims the benefit of U.S.application Ser. No. 15/941,937 (now U.S. Pat. No. 10,740,975), entitled“Mobile Augmented Reality System,” filed Mar. 30, 2018, which is acontinuation of and claims the benefit of U.S. application Ser. No.15/465,400 (now U.S. Pat. No. 10,134,196), entitled “MOBILE AUGMENTEDREALITY SYSTEM”, filed Mar. 21, 2017, which is a continuation of andclaims the benefit of U.S. application Ser. No. 13/175,608 (now U.S.Pat. No. 9,600,933), entitled “MOBILE AUGMENTED REALITY SYSTEM”, filedJul. 1, 2011. U.S. application Ser. Nos. 15/941,937, 15/465,400 and13/175,608 are hereby incorporated by reference in their entireties.

FIELD

Embodiments of the invention generally pertain to mobile computingdevices and more specifically to augmenting live views captured by imagedevices with related content.

BACKGROUND

Mobile computing devices typically include cameras, location andorientation sensors, and increasingly powerful computational resources.Mobile computing devices are also able to establish high bandwidthconnections to utilize cloud computing infrastructures and serviceproviders.

Displays included in mobile computing devices may be used as a liveviewfinder, allowing device users to capture real-time image data (e.g.,pictures, videos); however, most applications fail to utilize thecomputational resources available to the mobile computing device toprovide users with additional information relevant to the subject matterwithin the live view displayed (i.e., the viewfinder). The limitednumber of applications that do attempt to enhance the live view arelimited to displaying basic information such as the distance between theuser and a general location, or basic information about the user'ssurroundings (e.g., types of businesses within the live view) that isimprecise with respect to the user's position and with respect to therepresentation of the user's surroundings captured in the live view.

For example, if a user would like to identify a specific businesslocation within a captured live view, but said business is located in abuilding with several other businesses, current solutions may augmentthe live view with data identifying the building, but are incapable ofconveying more precise information about the business's location.

BRIEF DESCRIPTION OF THE DRAWINGS

The following description includes discussion of figures havingillustrations given by way of example of implementations of embodimentsof the invention. The drawings should be understood by way of example,and not by way of limitation. As used herein, references to one or more“embodiments” are to be understood as describing a particular feature,structure, or characteristic included in at least one implementation ofthe invention. Thus, phrases such as “in one embodiment” or “in analternate embodiment” appearing herein describe various embodiments andimplementations of the invention, and do not necessarily all refer tothe same embodiment. However, they are also not necessarily mutuallyexclusive.

FIG. 1 is a flow diagram of an embodiment of the invention.

FIG. 2 is a diagram of a 3D model used to augment image data accordingto an embodiment of the invention.

FIG. 3 is a live view scene to be augmented according to an embodimentof the invention.

FIG. 4 is an illustration of a plurality of projections masks generatedbased on 3D cloud data according to an embodiment of the invention.

FIG. 5 is an illustration of an augmented live view according to anembodiment of the invention.

FIGS. 6A and 6B illustrate projection masks and an augmented live viewfor multiple points of interest according to an embodiment of theinvention.

FIG. 7 is a block diagram of a system that may utilize an embodiment ofthe invention.

Descriptions of certain details and implementations follow, including adescription of the figures, which may depict some or all of theembodiments described below, as well as discussing other potentialembodiments or implementations of the inventive concepts presentedherein. An overview of embodiments of the invention is provided below,followed by a more detailed description with reference to the drawings.

DETAILED DESCRIPTION

Embodiments of the invention relate to systems, apparatuses and methodsto provide image data, augmented with related data, to be displayed on amobile computing device. While current solutions process 3D data toperform limited real-time object recognition, such information may beinsufficient to convey useful information to a user. For example, if auser would like to identify a specific business location within acaptured live view, and said business is located in a building withseveral other businesses, processing 3D point cloud data alone would notgenerate information sufficient to tell the user where the businesslocation is; in other words, current solutions may augment a live viewwith data identifying a building, but current solutions are incapable ofconveying more precise information about a business location in thescenario described above.

Embodiments of the invention address the above described limitations byaugmenting a live view with information identifying an object amongstother objects within 3D point cloud data—e.g., finding the extent of astorefront (or storefronts) that are included in a common buildingfacade. Embodiments of the invention may utilize other related data,such as image data and location data related to the object, to obtain aspecific location of an object within the live view.

Embodiments of the invention may further display a live view withaugmented data three-dimensionally consistent with the position andorientation of the image sensor of the mobile computing device.Embodiments of the invention may determine the alignment of the liveview by processing the live view image data and pre-existing 3D imagedata. The determination of the position and orientation of the imagesensor may further be based on sensor data from other sensors of thehost mobile computing device.

Example scenarios described below focus on points of interest withinbuildings that include multiple businesses or storefronts. It is to beunderstood that embodiments of the invention may be used for any pointof interest that is included in an object but not distinguishable from3D point cloud data alone, e.g., an object defined by a set of contoursincluded in a wall, a section of books included in a bookcase, aparticular feature of a sculpture, etc.

FIG. 1 is a flow diagram of an embodiment of the invention. Flowdiagrams as illustrated herein provide examples of sequences of variousprocess actions. Although shown in a particular sequence or order,unless otherwise specified, the order of the actions can be modified.Thus, the illustrated implementations should be understood only asexamples, and the illustrated processes can be performed in a differentorder, and some actions may be performed in parallel. Additionally, oneor more actions can be omitted in various embodiments of the invention;thus, not all actions are required in every implementation. Otherprocess flows are possible.

In process flow 100, a database of 3D cloud data for an object isprocessed, 110. Said database may include 3D point cloud data obtainedby lidar, stereo imaging, structured light, time-of-flight cameras, orany other technology used as inputs for generating 3D models of objects.It is to be understood that a 3D model of an object will includemultiple segments representing various contours of the object. Forexample, a 3D model of an object having a simple cube shape will includean arrangement of plurality of 2D planes, while objects having morecomplex surface variations may have an arrangement of both 2D planes and3D segments.

Thus, processing 3D data of an object may include extracting objectsegments (e.g., 2D planes), combining sets of point cloud data,associating point cloud data with location data (described below) andnoise removal. Said processing 3D data may be performed in differentorders via different processes.

In some embodiments, processing said 3D data may further includeprocessing pre-stored multiple 2D images of an object or scene fromvarious view angles along with corresponding 3D point cloud data setsfrom different view angles and view positions. These point cloud datasets may overlap. To identify the overlapping regions, the sets of pointcloud data may be restricted to a local neighborhood, and related pointcloud information may be merged.

In one embodiment, 3D point cloud data sets are modeled as a set ofnoisy measurements (the sources of noise may include uncertainty of thepoint of measurement, its orientation, sensor noise, etc) of the actual3D points. An estimate of the noise statistics and the original set of3D points in the overlapping regions may then be obtained. These noisestatistics may be utilized to obtain better estimates of the 3D pointsfor all point cloud data sets in order to extract projection masks,which will be utilized as described below. It is to be understood thatthe 3D points in the region of overlap have the most accurate estimates,and thus these points may be given priority in estimating the projectionmasks to which they belong (for example weighted least squares may beused for an estimation of the projection mask parameters with moreweights assigned to these points).

In one embodiment, confidence/reliability information associated with 3Dpoint cloud data is received. This additional information may beleveraged with the 3D point cloud data to infer the reliability for 3Dpoints' measurements. In one embodiment, color information may be usedto more accurately identify objects from 3D point cloud data. Forexample, 3D points that are within the same plane, are close in pixeldistance, and have similar colors (i.e., correspond to the same coloredpixels in pre-stored images) will tend to be assigned to the samepotential object.

Real-time object detection within a live view captured by an imagesensor of mobile computing device may be executed, 120. As describedabove, object segments extracted from the 3D point cloud data form arough 3D model of the pre-stored image data. These object segments fromthe 3D model may be used to identify the object within the captured liveview. In one embodiment, related data such as data identifying thelocation of the mobile computing device may also be used to helpidentify the object. A mobile computing device may utilize systemcomponents and applications (e.g., the Global Positioning System (GPS)sensors, cellular or WiFi network connections, orientation sensors) tonarrow down the device's location.

The position and orientation of the image sensor of the mobile computingdevice, with respect to the object, is determined, 130. Said positionand orientation may be determined based on the captured live view of theobject—i.e., by identifying which object segments are visible in thelive view and matching those object segments to the 3D model of theobject, it can be determined where the image sensor of the mobilecomputing device is positioned with respect to the object. Returning tothe example of viewing a cube-shaped object, if only a north-facing sideof the cube-shaped object is visible in the live view, it can bedetermined that the image sensor is positioned north of the cube-shapedobject, and close-enough in proximity to the object such that only oneside of the object is visible in the live view. In one embodiment, theassociated location data described above is also used to help determinethe position and view angle of the image sensor.

A projection mask for a point of interest included in the object isgenerated from the 3D model of the object, 140. A point of interest maybe identified, for example, by name, by category, by subject matter,etc. As described above, a particular challenge in augmenting an objectto highlight a point of interest is when the object also includes otherpotential points of interest. Thus, multiple potential points ofinterest will be included within the same 3D model of the object, andmay also be included within the same 3D segment of the 3D model or evenwithin the same 2D plane of the 3D model.

In one embodiment, the location and orientation information of the imagesensor, along with related pre-stored image data of the approximatelocation of the point of interest whose views are close to theapproximate view (e.g., reference models), may be utilized to extractvisual features of the object. These visual features are mapped to theappropriate segments of the 3D model, and projection masks for the pointof interest in the 3D model of the object are formed. Thus, eachprojection mask will include a subset of segments of the 3D model, or insome scenarios, a portion of a segment of the 3D model as describedbelow.

The live view is then augmented with image data, such that the imagedata is included in the projection mask and is displayedthree-dimensionally consistent with the position and the orientation ofthe image sensor, 150. Thus, augmentation applications or modules mayproject data within the generated projection mask that is relevant tothe point or object of interest. In other words, the user may take animage with the mobile computing device and register it in the correctperspective in the 3D model. The image or live view may be augmented inreal time with additional information, such as text, images or videos,or some 3D structure, added in the correct perspective.

Mobile client platforms that execute the above described embodimentprocess may include cameras, location and orientation sensors, andwireless communication capabilities. In one embodiment a mobile clientdevice accesses processed data from a server through a network inreal-time; if the user cannot connect to a network due to costs or otherreasons, a small database can be pre-loaded on the mobile client device.

Thus, embodiments of the invention may execute off-line databaseprocessing, on-line object detection and mobile computing devicetracking, and combine the off-line and on-line data into a meaningfulaugmented user display.

FIG. 2 is a diagram of a 3D model used to augment image data accordingto an embodiment of the invention. FIG. 2 illustrates image data, 3Dpoint cloud data and projection masks that may be processed byembodiments of the invention. 3D point cloud 250 corresponds topre-stored 3D image 200 of object 205. Each 3D point in cloud 250represents the actual 3D location of a pixel of image 200. Some pixelsin image 200 may not have a corresponding 3D point, e.g., pixels in thesky.

In some embodiments 3D point cloud data 250 is transformed into acoordinate system more suitable for subsequent processing. For example,if 3D point cloud data 250 is in the format of “latitude, longitude, andaltitude,” it may be more useful to transform the format to a localcoordinate system such as east, north, and up (ENU), so the values ofthe coordinates are smaller (i.e., compared to coordinates in a systemwith the center of the earth as the origin). This transformation mayalso better convey the vertical and horizontal orientations of 3D pointcloud data 250.

In some embodiments, 3D point cloud data 250 results from a larger setof 3D point cloud data sub-sampled to make computation faster. This maybe accomplished by downsampling the pre-stored image or thecorresponding 3D point cloud. For example, for a 200×500 image, the sizeof the corresponding 3D point cloud data may consist of up to 150,000points. By sampling the image at a rate 10 in both horizontal andvertical dimensions, the number of 3D points may be reduced to 1,500points.

In order to determine the contours of object 205 from 3D point clouddata 250, embodiments of the invention may adopt a random sampleconsensus (RANSAC) approach and combine both pre-stored image 200 and 3Dpoint cloud data 250 to guide the sampling process in RANSAC rather thanarbitrarily sampling the 3D point cloud data. In this example, becauseobject 205 is cube shaped, its 3D model is made from a limited number of2D planes, including 2D planar segments 290, 291 and 292.

For the example use of identifying a user identified point of interestin object 205, it may be determined which of 2D planar segments includesaid point of interest. In this example, the user identified point ofinterest is visible in 2D planar segments 290 and 291. Other informationof the user selected point of interest is processed to determine itsboundaries in order to generate projection mask 299 as shown.

FIG. 3 is a live view scene to be augmented according to an embodimentof the invention. In this example, display 310 includes real-time view320 of a user's surroundings via an image sensor (e.g., a camera)included in mobile computing device 300. Said device may be any portablecomputing system, such as a laptop computer, a tablet computer, asmartphone, a hand-held computing device, a vehicle computing system,etc.

It is to be understood that by displaying real-time view 320, display310 may function as a viewfinder displaying image data (e.g., pictures,video) to allow a user to observe the target of the image sensor ofdevice 300. In this example, real-time view 320 includes a view ofbuildings 330, 340, 350, 360 and 370, wherein some of the views of thebuildings are partially obstructed by trees 380 and 385.

Embodiments of the invention may process available 3D models andpotential augmenting information for use on mobile device 300 inreal-time. For example, embodiments of the invention may merge streetaddresses, 3D models, 3D point-clouds, depth images, and radiometric(color) images to determine the true extent of potential points ofinterest within each building. Because some embodiments preprocess thisinformation offline, there are no strict time, computational complexityor storage constraints (i.e., it is understood that building detectionis a computationally expensive operation and cannot typically beperformed at video frame rates).

Furthermore, the above described processing enables device 300 to detect(i.e., recognize) and track the buildings in view 320, and estimate theposition and orientation of the image sensor of device 300 relative tothe buildings. In one embodiment, GPS, or other location services suchas WiFi and 3G, coupled with orientation sensors (e.g., a compass, anaccelerometer) are used to find the image sensors location andorientation. It is to be understood that this sensor information maysimplify building detection because the system need only match againstdatabase images that fulfill location and orientation constraints. Toperform building detection, visual features from input video frames ondevice 300 may be extracted and compared to pre-processed visualfeatures from candidate database images.

Because building detection and device tracking processes may takedifferent lengths of time to compute, embodiments of the invention mayuse a multi-threaded framework to coordinate these tasks. For example, atracking thread for determining the location and orientation of device300 may be done in real-time, while a building detection thread may bedone without a hard time-constraint. Said multi-threaded framework mayalso coordinate the outputs from the building detection thread and thetracking thread. In one embodiment, a confidence measure is utilized tocheck the reliability of output from building detection processes, andwhen new results are available, display 310 is updated if they are moreconfident than prior results. The output from the building detectionthread is used to update the determined pose of device 300 if theconfidence value is larger than the aged confidence value from thetracking thread.

Thus, prior to augmenting live view 320, embodiments of the inventionmay execute the building detection and device tracking processesdescribed above. The processing may take place on the mobile computingdevice, on another computing device or server operatively coupled to themobile computing device, or any combination thereof. Similarly, the dataused in the building detection and device tracking processes may beincluded on the mobile computing device, on another computing device orserver operatively coupled to the mobile computing device, or anycombination thereof.

In this example, the position of device 300 is such that only the frontfacade of buildings 330-370 are in view 320. As described above, aparticular set of challenges arises in outdoor building recognition andaugmentation when a user identified point of interest is a businessincluded in a building with other unselected businesses. In thisexample, buildings 330 and 350 each include a plurality of businesses;thus merely identifying said buildings and extracting projection masksrelated to the front facade of each building is insufficient to conveymeaningful augmented content to the user. For these reasons the systemmust be aware of the each building's 3D geometry, as well as itslatitude and longitude. When the scene is augmented, the labels must bemeaningfully related to the individual store-fronts. Thus, embodimentsof the invention may further process related data to enable localizingaugmented content to the relevant store-front, and generate projectionmasks such that the augmented content appears in the appropriate sectionof the building facade. In one embodiment, the augmented content is tobe located at or near the center of the store-front, and the extent ofthe generated projection mask will be the same as that of thestorefront. To present this type of experience to the user, embodimentsof the invention may semantically segment buildings when appropriate.

FIG. 4 is an illustration of a plurality of projections masks generatedbased on 3D cloud data according to an embodiment of the invention. Theprojection masks illustrated are extracted from 3D information relatedto view 320 through processes described above. In this example, giventhe shape of the buildings and their perspective within view 310, saidprojection masks are shown to be 2D planar segments included in thefront facade of each building.

Building 330 includes four different potential points of interest (e.g.,four different businesses) and is represented by projection masks 431,432, 433 and 434. Building 350 includes three different potential pointsof interest, and is represented by projection masks 451, 452 and 453.Buildings 340, 360 and 370 each only contain one potential point ofinterest, and thus each is represented by single projection masks 441,461 and 471 respectively.

Each of the above described projection masks is geometrically consistentwith a surface plane of their respective buildings as they appear inview 320. For buildings 330 and 350, which include multiple potentialpoints of interest, embodiments of the invention may process locationinformation, building geometry, and visual imagery to generate theappropriate projection masks.

In one embodiment, the center of the store-front in the building's 3Dmodel is estimated. This center may be estimated with the assistance of,for example, each business's known street address, which can give anordering and approximate location of each store-front relative to thestreet block. This coarse estimate can then be projected onto the 3Dmodel, where geometric constraints are enforced. Such constraints mayinclude minimum and maximum widths for store-fronts, or positionsrelative to distinct planes on the facade.

In one embodiment, processed data includes a collection of street-levelspherical panorama images at known dense GPS locations. These imagesprocessed together with the 3D information in the form of depth data andsequence of projection masks will approximate the structure of the urbanscene included in view 320. As described above, in some embodimentspreprocessing is necessary to make such data usable for processing onthe mobile device (e.g., detection, tracking, and augmentation). Thesepanoramas are processed by projecting them on to the above describedplanes at fixed orientations relative to the device including display310. It is to be understood that such projected 2D images may besuitable to be used as reference images for visual matching on a mobiledevice when they have similar geometry to query images from the mobiledevice.

In one embodiment, further estimating the true extent of the individualstore-fronts within each building is done using the above describedcollection of street-level spherical panorama images. Each store-frontis represented in multiple views in the database, from which afine-scale depth is reconstructed. This color and depth data may then becomposed with the 3D model, and each store-front's extent can be grownfrom the central point until a natural color and/or depth boundary isreached.

Embodiments of the invention may further use the location information,for example, the GPS coordinates of the centers of the projection masks,to perform geocoding (adding a coordinate to a planar image segment), orinverse geocoding (looking up existing information for a givenlocation). Inverse geocoding can be used as one source of the augmentinginformation for real-time device operation. Once a geocoding system isin place, it may be deployed as a service to geotag legacy image andvideo content for the user for subsequent use.

Estimating the extent of a store-front can be further aided by theinclusion of more data provided by the business owner, or device user.In the case of the device user, embodiments of the invention may providea simple editing method for manipulating the augmentation. This may be asimple polygon, whose corners the user is allowed to move. This willallow a user to input a correct, but slightly misaligned result toquickly update the service. These updates are then aggregated andcompared against each other and the other known data so that the nextuser will get an improved service. In one embodiment, because it isassumed a store owner is a more motivated and trusted contributor ofaccurate data about the store, data from the business owner may be givena higher priority in the aggregation scheme.

FIG. 5 is an illustration of an augmented live view according to anembodiment of the invention. In this embodiment, augmented contentwindow 500 identifies the appropriate point of interest in building 350and displays image data 510 related to the point of interest inreal-time view 520. Image data 510 is displayed geometrically consistentwith its respective projection mask (i.e., geometrically consistent with2D plane 452 of FIG. 4).

Image data 510 may include an image related to the point of interest,for example an image identifying the boundaries of the point ofinterest, video content related to the point of interest (e.g., a videoadvertisement), 3D animated object (e.g., 3D animated advertisement), ortext data related to the point of interest. Said image data may beincluded in memory storage of device 300, or obtained from a databasevia a network connection (e.g., wireless internet connection).

FIGS. 6A and 6B illustrate projection masks and an augmented live viewfor multiple points of interest according to an embodiment of theinvention. In this embodiment, the captured view of building 600includes two facade segments 610 and 630 that, from the perspective ofthe image sensor, are relatively planar, and contoured facade segment620. Thus, embodiments of the invention may utilize 2D planar segmentsand 3D contoured/spherical masks to provide the system with a coarseestimate of the extent of the facade of building 600.

Thus, it is to be understood that projected masks 610-630 may be used torepresent the surface contours of the relevant portions of an object.Contoured projection mask 620 may be used for augmentation content laterin real-time processing, wherein the augmentation content will beprojected consistent with the contour of mask 620 (as well as consistentwith the orientation of the host mobile device with respect to building600). Additionally, contour information may be used in combination withedge detection to improve object detection to identify building 600.

Thus, multiple points of interest may be identified. Related image datacorresponding to the points of interest (e.g., an icon, text data,video, 3D animated object, or image data as described above) may bedisplayed. In this example, text data identifying the name of eachbusiness in building 600 is displayed as shown in FIG. 6B. The text datais both centered within the boundaries of each respective business, anddisplayed based on the relevant projected mask.

FIG. 7 is a block diagram of a system that may utilize an embodiment ofthe invention. System 700 may include pre-processing module 710 toexecute the operations involving pre-stored images and 3D point clouddata to generate 3D models of objects as described above. System 700 mayfurther include image sensor 720 to capture image data as describedabove.

Projection module 730 may process 3D data and location data of an objectto generate a projection mask, as described above. Augmentation module740 may augment the captured image data with associated content withinthe projection mask as described above. The augmented image data may bedisplayed on display 750.

In this embodiment, modules 710, 730 and 740 are executed via processor760. All components of system 700 described above may be operativelycoupled via bus 770. It is to be understood that the various modulesdescribed in FIG. 7 may all be included in a mobile computing device, orseparately in various locations (i.e., any or all of the modules in FIG.7 may be included in a server interfaced with a mobile computer deviceto provide “backend processing”). Furthermore, it is to be understoodthe operations related to the described modules are an exampleembodiment only, and that any operations described above may be executedvia multiple devices operatively coupled together.

Various components referred to above as processes, servers, or toolsdescribed herein may be a means for performing the functions described.Each component described herein includes software or hardware, or acombination of these. Each and all components may be implemented assoftware modules, hardware modules, special-purpose hardware (e.g.,application specific hardware, ASICs, DSPs, etc.), embedded controllers,hardwired circuitry, hardware logic, etc. Software content (e.g., data,instructions, configuration) may be provided via an article ofmanufacture including a non-transitory, tangible computer or machinereadable storage medium, which provides content that representsinstructions that can be executed. The content may result in a computerperforming various functions/operations described herein. A computerreadable storage medium includes any mechanism that provides (i.e.,stores and/or transmits) information in a form accessible by a computer(e.g., computing device, electronic system, etc.), such asrecordable/non-recordable media (e.g., read only memory (ROM), randomaccess memory (RAM), magnetic disk storage media, optical storage media,flash memory devices, etc.). The content may be directly executable(“object” or “executable” form), source code, or difference code(“delta” or “patch” code). A computer readable storage medium may alsoinclude a storage or database from which content can be downloaded. Acomputer readable medium may also include a device or product havingcontent stored thereon at a time of sale or delivery. Thus, delivering adevice with stored content, or offering content for download over acommunication medium may be understood as providing an article ofmanufacture with such content described herein.

What is claimed is:
 1. A mobile device, comprising: at least one camerato capture a live view of a real-world scene, the real-world sceneincluding a feature of interest, the feature of interest captured in thelive view; at least one sensor to facilitate determination of a positionand an orientation of the at least one camera relative to the scene;instructions in the mobile device; at least one processor to execute theinstructions to: identify a size and position of the feature of interestin the live view of the scene from a perspective of the at least onecamera; and identify a plane in the live view of the scene, the planehaving a first area in the live view of the scene; and a screen topresent the live view of the scene and augmented reality data over thelive view of the scene, the augmented reality data to be presented in asecond area corresponding to the size and position of the feature ofinterest, the second area smaller than and within the first area.
 2. Themobile device as defined in claim 1, wherein the live view correspondsto a perspective of the at least one camera at a time when the augmentedreality data is presented on the screen.
 3. The mobile device as definedin claim 1, wherein the augmented reality data includes an image to bedisplayed consistent with a three-dimensional perspective of the atleast one camera corresponding to the position and the orientation ofthe at least one camera relative to the scene.
 4. The mobile device asdefined in claim 1, wherein the at least one processor is to define aboundary for the second area.
 5. The mobile device as defined in claim4, wherein the augmented reality data includes an identification of theboundary of the second area.
 6. The mobile device as defined in claim 1,wherein the at least one sensor includes at least one of anaccelerometer or a compass.
 7. The mobile device as defined in claim 1,wherein the at least one sensor is to collect global position satellitedata.
 8. At least one storage device comprising instructions that, whenexecuted, cause at least one machine to at least: cause at least onecamera to capture a live view of a real-world scene, the real-worldscene including a feature of interest, the feature of interest capturedin the live view; determine, based on a signal from at least one sensor,a position and an orientation of the at least one camera relative to thescene; identify a size and position of the feature of interest in thelive view of the scene from a perspective of the at least one camera;identify a plane in the live view of the scene, the plane having a firstarea in the live view of the scene; generate augmented reality data topresent on a screen over the live view of the scene; and causepresentation of the live view of the scene and the augmented realitydata in a second area corresponding to the size and position of thefeature of interest, the second area smaller than and within the firstarea.
 9. The at least one storage device as defined in claim 8, whereinthe live view corresponds to a perspective of the at least one camera ata time when the augmented reality data is presented on the screen. 10.The at least one storage device as defined in claim 8, wherein theinstructions cause the at least one machine to generate the augmentedreality data to be consistent with a three-dimensional perspective ofthe at least one camera from the position and the orientation of the atleast one camera relative to the scene.
 11. The at least one storagedevice as defined in claim 8, wherein the instructions cause the atleast one machine to define a boundary for the second area.
 12. The atleast one storage device as defined in claim 11, wherein the augmentedreality data includes an identification of the boundary of the secondarea.
 13. An apparatus, comprising: means for capturing a live view of areal-world scene, the real-world scene including a feature of interest,the feature of interest captured in the live view; means for sensing aposition and an orientation of the capturing means relative to thescene; and means for processing to: identify a size of the feature ofinterest in the live view of the scene from a perspective of thecapturing means; identify a plane in the live view of the scene, theplane having a first area in the live view of the scene; and generateaugmented reality data for display over the live view of the scene; andmeans for displaying the live view of the scene and the augmentedreality data in a second area corresponding to the size and position ofthe feature of interest, the second area smaller than and within thefirst area.
 14. The apparatus as defined in claim 13, wherein the liveview corresponds to a perspective of the capturing means at a time whenthe augmented reality data is displayed on the displaying means.
 15. Theapparatus as defined in claim 13, wherein the processing means is togenerate the augmented reality data to be consistent with athree-dimensional perspective of the capturing means corresponding tothe position and the orientation of the capturing means relative to thescene.
 16. The apparatus as defined in claim 13, wherein the processingmeans is to define a boundary for the second area.
 17. The apparatus asdefined in claim 16, wherein the augmented reality data includes anidentification of the boundary of the second area.
 18. The apparatus asdefined in claim 13, wherein the sensing means includes at least one ofan accelerometer or a compass.
 19. The apparatus as defined in claim 13,wherein the sensing means is to generate global position satellite data.20. A method, comprising: causing at least one camera to capture a liveview of a real-world scene, the real-world scene including a feature ofinterest, the feature of interest captured in the live view;determining, based on a signal from at least one sensor, a position andan orientation of the at least one camera relative to the scene;identifying a size of the feature of interest in the live view of thescene from a perspective of the at least one camera; identifying a planein the live view of the scene, the plane having a first area in the liveview of the scene; generating, by executing an instruction with at leastone processor, augmented reality data to present on a screen over thelive view of the scene; and causing presentation of the live view of thescene and the augmented reality data in a second area corresponding tothe size and position of the feature of interest, the second areasmaller than and within the first area.
 21. The method as defined inclaim 20, wherein the live view corresponds to a perspective of the atleast one camera at a time when the augmented reality data is presentedon the screen.
 22. The method as defined in claim 20, wherein thegeneration of the augmented reality data includes generating theaugmented reality data to be consistent with a three-dimensionalperspective of the at least one camera from the position and theorientation of the at least one camera relative to the scene.
 23. Themethod as defined in claim 20, further including defining a boundary forthe second area.
 24. The method as defined in claim 23, wherein theaugmented reality data includes an identification of the boundary of thesecond area.